Light sensitive display

ABSTRACT

A light sensitive display.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation of U.S. patent application Ser. No.11/407,545, filed Apr. 19, 2006, which is a continuation of U.S. patentapplication Ser. No. 10/442,433, filed May 20, 2003, now U.S. Pat. No.7,053,967, which application claims the benefit of U.S. Provisional App.No. 60/383,040, filed May 23, 2002.

BACKGROUND OF THE INVENTION

The present invention relates to touch sensitive displays.

Touch sensitive screens (“touch screens”) are devices that typicallymount over a display such as a cathode ray tube. With a touch screen, auser can select from options displayed on the display's viewing surfaceby touching the surface adjacent to the desired option, or, in somedesigns, touching the option directly. Common techniques employed inthese devices for detecting the location of a touch include mechanicalbuttons, crossed beams of infrared light, acoustic surface waves,capacitance sensing, and resistive materials.

For example, Kasday, U.S. Pat. No. 4,484,179 discloses anoptically-based touch screen comprising a flexible clear membranesupported above a glass screen whose edges are fitted with photodiodes.When the membrane is flexed into contact with the screen by a touch,light which previously would have passed through the membrane and glassscreen is trapped between the screen surfaces by total internalreflection. This trapped light travels to the edge of the glass screenwhere it is detected by the photodiodes which produce a correspondingoutput signal. The touch position is determined by coordinating theposition of the CRT raster beam with the timing of the output signalsfrom the several photodiodes. The optically-based touch screen increasesthe expense of the display, and increases the complexity of the display.

Denlinger, U.S. Pat. No. 4,782,328 on the other hand, relies onreflection of ambient light from the actual touch source, such as afinger or pointer, into a pair of photosensors mounted at corners of thetouch screen. By measuring the intensity of the reflected light receivedby each photosensor, a computer calculates the location of the touchsource with reference to the screen. The inclusion of the photosensorsand associated computer increases the expense of the display, andincreases the complexity of the display.

May, U.S. Pat. No. 5,105,186, discloses a liquid crystal touch screenthat includes an upper glass sheet and a lower glass sheet separated byspacers. Sandwiched between the glass sheets is a thin layer of liquidcrystal material. The inner surface of each piece of glass is coatedwith a transparent, conductive layer of metal oxide. Affixed to theouter surface of the upper glass sheet is an upper polarizer whichcomprises the display's viewing surface. Affixed to the outer surface ofglass sheet is a lower polarizer. Forming the back surface of the liquidcrystal display is a transflector adjacent to the lower polarizer. Atransflector transmits some of the light striking its surface andreflects some light. Adjacent to transflector is a light detecting arrayof light dependent resistors whose resistance varies with the intensityof light detected. The resistance increases as the light intensitydecreases, such as occurs when a shadow is cast on the viewing surface.The light detecting array detect a change in the light transmittedthrough the transflector caused by a touching of viewing surface.Similar to touch sensitive structures affixed to the front of the liquidcrystal stack, the light sensitive material affixed to the rear of theliquid crystal stack similarly pose potential problems limiting contrastof the display, increasing the expense of the display, and increasingthe complexity of the display.

Touch screens that have a transparent surface which mounts between theuser and the display's viewing surface have several drawbacks. Forexample, the transparent surface, and other layers between the liquidcrystal material and the transparent surface may result in multiplereflections which decreases the display's contrast and produces glare.Moreover, adding an additional touch panel to the display increases themanufacturing expense of the display and increases the complexity of thedisplay. Also, the incorporation of the touch screen reduces the overallmanufacturing yield of the display.

Accordingly, what is desired is a touch screen that does notsignificantly decrease the contrast ratio, does not significantlyincrease the glare, does not significantly increase the expense of thedisplay, and does not significantly increase the complexity of thedisplay.

The foregoing and other objectives, features, and advantages of theinvention will be more readily understood upon consideration of thefollowing detailed description of the invention, taken in conjunctionwith the accompanying drawings.

BRIEF DESCRIPTION OF THE SEVERAL VIEWS OF THE DRAWINGS

FIG. 1 is a cross sectional view of a traditional active matrix liquidcrystal display.

FIG. 2 is a schematic of the thin film transistor array.

FIG. 3 is a layout of the thin film transistor array of FIG. 2.

FIGS. 4A-4H is a set of steps suitable for constructing pixel electrodesand amorphous silicon thin-film transistors.

FIG. 5 illustrates pixel electrodes, color filters, and a black matrix.

FIG. 6 illustrates a schematic of the active matrix elements, pixelelectrode, photo TFT, readout TFT, and a black matrix.

FIG. 7 illustrates a pixel electrode, photo TFT, readout TFT, and ablack matrix.

FIG. 8 is a layout of the thin film transistor array of FIGS. 6 and 7.

FIG. 9 is a graph of the capacitive charge on the light sensitiveelements as a result of touching the display at high ambient lightingconditions.

FIG. 10 is a graph of the capacitive charge on the light sensitiveelements as a result of touching the display at low ambient lightingconditions.

FIG. 11 is a graph of the photo-currents in an amorphous silicon TFTarray.

FIG. 12 is a graph of the capacitive charge on the light sensitiveelements as a result of touching the display and providing light from alight wand.

FIG. 13 is an alternative layout of the pixel electrodes.

FIG. 14 illustrates a timing set for the layout of FIG. 13.

FIG. 15 illustrates a handheld device together with an optical wand.

FIG. 16 illustrates even/odd frame addressing.

FIG. 17 illustrates a front illuminated display.

FIG. 18 illustrates total internal reflections.

FIG. 19 illustrates a small amount of diffraction of the propagatinglight.

FIG. 20 illustrates significant diffraction as a result of a plasticpen.

FIG. 21 illustrates a shadow of a pointing device and a shadow withilluminated region of a pointing device.

DETAILED DESCRIPTION OF PREFERRED EMBODIMENT

Referring to FIG. 1, a liquid crystal display (LCD) 50 (indicated by abracket) comprises generally, a backlight 52 and a light valve 54(indicated by a bracket). Since liquid crystals do not emit light, mostLCD panels are backlit with fluorescent tubes or arrays oflight-emitting diodes (LEDs) that are built into the sides or back ofthe panel. To disperse the light and obtain a more uniform intensityover the surface of the display, light from the backlight 52 typicallypasses through a diffuser 56 before impinging on the light valve 54.

The transmittance of light from the backlight 52 to the eye of a viewer58, observing an image displayed on the front of the panel, iscontrolled by the light valve 54. The light valve 54 normally includes apair of polarizers 60 and 62 separated by a layer of liquid crystals 64contained in a cell gap between glass or plastic plates, and thepolarizers. Light from the backlight 52 impinging on the first polarizer62 comprises electromagnetic waves vibrating in a plurality of planes.Only that portion of the light vibrating in the plane of the opticalaxis of the polarizer passes through the polarizer. In an LCD lightvalve, the optical axes of the first 62 and second 60 polarizer aretypically arranged at an angle so that light passing through the firstpolarizer would normally be blocked from passing through the secondpolarizer in the series. However, the orientation of the translucentcrystals in the layer of liquid crystals 64 can be locally controlled toeither “twist” the vibratory plane of the light into alignment with theoptical axes of the polarizer, permitting light to pass through thelight valve creating a bright picture element or pixel, or out ofalignment with the optical axis of one of the polarizes, attenuating thelight and creating a darker area of the screen or pixel.

The surfaces of the a first glass substrate 61 and a second glasssubstrate 63 form the walls of the cell gap are buffed to producemicroscopic grooves to physically align the molecules of liquid crystal64 immediately adjacent to the walls. Molecular forces cause adjacentliquid crystal molecules to attempt to align with their neighbors withthe result that the orientation of the molecules in the column ofmolecules spanning the cell gap twist over the length of the column.Likewise, the plane of vibration of light transiting the column ofmolecules will be “twisted” from the optical axis of the first polarizer62 to a plane determined by the orientation of the liquid crystals atthe opposite wall of the cell gap. If the wall of the cell gap is buffedto align adjacent crystals with the optical axis of the secondpolarizer, light from the backlight 52 can pass through the series ofpolarizers 60 and 62 to produce a lighted area of the display whenviewed from the front of the panel (a “normally white” LCD).

To darken a pixel and create an image, a voltage, typically controlledby a thin film transistor, is applied to an electrode in an array oftransparent electrodes deposited on the walls of the cell gap. Theliquid crystal molecules adjacent to the electrode are attracted by thefield produced by the voltage and rotate to align with the field. As themolecules of liquid crystal are rotated by the electric field, thecolumn of crystals is “untwisted,” and the optical axes of the crystalsadjacent to the cell wall are rotated progressively out of alignmentwith the optical axis of the corresponding polarizer progressivelyreducing the local transmittance of the light valve 54 and attenuatingthe luminance of the corresponding pixel. In other words, in a normallywhite twisted nematic device there are generally two modes of operation,one without a voltage applied to the molecules and one with a voltageapplied to the molecules. With a voltage applied (e.g., driven mode) tothe molecules the molecules rotate their polarization axis which resultsin inhibiting the passage of light to the viewer. Similarly, without avoltage applied (e.g., non-driven mode) the polarization axis is notrotated so that the passage of light is not inhibited to the viewer.

Conversely, the polarizers and buffing of the light valve can bearranged to produce a “normally black” LCD having pixels that are dark(light is blocked) when the electrodes are not energized and light whenthe electrodes are energized. Color LCD displays are created by varyingthe intensity of transmitted light for each of a plurality of primarycolor (typically, red, green, and blue) sub-pixels that make up adisplayed pixel.

The aforementioned example was described with respect to a twistednematic device. However, this description is only an example and otherdevices may likewise be used, including but not limited to, multi-domainvertical alignment, patterned vertical alignment, in-plane switching,and super-twisted nematic type LCDs. In addition other devices, such asfor example, plasma displays, organic displays, active matrix organiclight emitting display, electroluminescent displays, liquid crystal onsilicon displays, reflective liquid crystal devices may likewise beused. For such displays the light emitting portion of the display, orportion of the display that permits the display of selected portions oflight may be considered to selectively cause the pixels to providelight.

For an active matrix LCD (AMLCD) the inner surface of the second glasssubstrate 63 is normally coated with a continuous electrode while thefirst glass substrate 61 is patterned into individual pixel electrodes.The continuous electrode may be constructed using a transparentelectrode, such as indium tin oxide. The first glass substrate 61 mayinclude thin film transistors (TFTs) which act as individual switchesfor each pixel electrode (or group of pixel electrodes) corresponding toa pixel (or group of pixels). The TFTs are addressed by a set ofmultiplexed electrodes running along the gaps between the pixelelectrodes. Alternatively the pixel electrodes may be on a differentlayer from the TFTs. A pixel is addressed by applying voltage (orcurrent) to a selected line, which switches the TFT on and allows chargefrom the data line to flow onto the rear pixel electrodes. Thecombination of voltages between the front electrode and the pixelelectrodes sets up a voltage across the pixels and turns the respectivepixels on. The thin-film transistors are typically constructed fromamorphous silicon, while other types of switching devices may likewisebe used, such as for example, metal-insulator-metal diode andpolysilicon thin-film transistors. The TFT array and pixel electrodesmay alternatively be on the top of the liquid crystal material. Also,the continuous electrode may be patterned or portions selectivelyselected, as desired. Also the light sensitive elements may likewise belocated on the top, or otherwise above, of the liquid crystal material,if desired.

Referring to FIG. 2, the active matrix layer may include a set of datalines and a set of select lines. Normally one data line is included foreach column of pixels across the display and one select line is includedfor each row of pixels down the display, thereby creating an array ofconductive lines. To load the data to the respective pixels indicatingwhich pixels should be illuminated, normally in a row-by-row manner, aset of voltages are imposed on the respective data lines 204 whichimposes a voltage on the sources 202 of latching transistors 200. Theselection of a respective select line 210, interconnected to the gates212 of the respective latching transistors 200, permits the voltageimposed on the sources 202 to be passed to the drain 214 of the latchingtransistors 200. The drains 214 of the latching transistors 200 areelectrically connected to respective pixel electrodes and arecapacitively coupled to a respective common line 221 through arespective Cst capacitor 218. In addition, a respective capacitanceexists between the pixel electrodes enclosing the liquid crystalmaterial, noted as capacitances Clc 222 (between the pixel electrodesand the common electrode on the color plate). The common line 221provides a voltage reference. In other words, the voltage data(representative of the image to be displayed) is loaded into the datalines for a row of latching transistors 200 and imposing a voltage onthe select line 210 latches that data into the holding capacitors andhence the pixel electrodes.

Referring to FIG. 3, a schematic layout is shown of the active matrixlayer. The pixel electrodes 230 are generally grouped into a “single”effective pixel so that a corresponding set of pixel electrodes 230 maybe associated with respective color filters (e.g., red, green, blue).The latching transistors 200 interconnect the respective pixelelectrodes 230 with the data lines and the select line. The pixelelectrodes 230 may be interconnected to the common line 221 by thecapacitors Cst 218.

Referring to FIG. 4, the active matrix layer may be constructed using anamorphous silicon thin-film transistor fabrication process. The stepsmay include gate metal deposition (FIG. 4A), a photolithography/etch(FIG. 4B), a gate insulator and amorphous silicon deposition (FIG. 4C),a photolithography/etch (FIG. 4D), a source/drain metal deposition (FIG.4E), a photolithography/etch (FIG. 4F), an ITO deposition (FIG. 4G), anda photolithography/etch (FIG. 4H). Other processes may likewise be used,as desired.

The present inventors considered different potential architectural touchpanel schemes to incorporate additional optical layers between thepolarizer on the front of the liquid crystal display and the front ofthe display. These additional layers include, for example, glass plates,wire grids, transparent electrodes, plastic plates, spacers, and othermaterials. In addition, the present inventors considered the additionallayers with different optical characteristics, such as for example,birefringence, non-birefringence, narrow range of wavelengths, widerange of wavelengths, etc. After an extensive analysis of differentpotential configurations of the touch screen portion added to thedisplay together with materials having different optical properties andfurther being applied to the different types of technologies (e.g.,mechanical switches, crossed beams of infrared light, acoustic surfacewaves, capacitance sensing, and resistive membranes), the presentinventors determined that an optimized touch screen is merely a tradeoffbetween different undesirable properties. Accordingly, the design of anoptimized touch screen is an ultimately unsolvable task. In contrast todesigning an improved touch screen, the present inventors came to therealization that modification of the structure of the active matrixliquid crystal device itself could provide an improved touch screencapability without all of the attendant drawbacks to the touch screenconfiguration located on the front of the display.

Referring to FIG. 5, with particular attention to the latchingtransistors of the pixel electrodes, a black matrix 240 is overlying thelatching transistors so that significant ambient light does not strikethe transistors. Color filters 242 may be located above the pixelelectrodes. Ambient light striking the latching transistors results indraining the charge imposed on the pixel electrodes through thetransistor. The discharge of the charge imposed on the pixel electrodesresults in a decrease in the operational characteristics of the display,frequently to the extent that the display is rendered effectivelyinoperative. With the realization that amorphous silicon transistors aresensitive to light incident thereon, the present inventors determinedthat such transistors within the active matrix layer may be used as abasis upon which to detect the existence of or non-existence of ambientlight incident thereon (e.g., relative values thereto).

Referring to FIG. 6, a modified active matrix layer may include aphoto-sensitive structure or elements. The preferred photo-sensitivestructure includes a photo-sensitive thin film transistor (photo TFT)interconnected to a readout thin film transistor (readout TFT). Acapacitor Cst2 may interconnect the common line to the transistors.Referring to FIG. 7, a black matrix may be in an overlying relationshipto the readout TFT. The black matrix is preferably an opaque material orotherwise the structure of the display selectively inhibiting thetransmission of light to selective portions of the active matrix layer.Preferably the black matrix is completely overlying the amorphoussilicon portion of the readout TFT, and at least partially overlying theamorphous silicon portion of the readout TFT. Preferably the blackmatrix is completely non-overlying the amorphous silicon portion of thephoto TFT, and at least partially non-overlying the amorphous siliconportion of the photo TFT. Overlying does not necessarily denote directcontact between the layers, but is intended to denote in the generalsense the stacked structure of materials. In some embodiments, the blackmatrix inhibits ambient light from impacting the amorphous siliconportion of the readout TFT to an extent greater than inhibiting ambientlight from impacting the amorphous silicon portion of the photo TFT.

As an example, the common line may be set at a negative voltagepotential, such as −10 volts. During the previous readout cycle, avoltage is imposed on the select line which causes the voltage on thereadout line to be coupled to the drain of the photo TFT and the drainof the readout TFT, which results in a voltage potential across Cst2.The voltage coupled to the drain of the photo TFT and the drain of thereadout TFT is approximately ground (e.g., zero volts) with thenon-inverting input of the operational amplifier connected to ground.The voltage imposed on the select line is removed so that the readoutTFT will turn “off”.

Under normal operational conditions ambient light from the front of thedisplay passes through the black matrix and strikes the amorphoussilicon of the photo TFT. However, if a person touches the front of thedisplay in a region over the opening in the black matrix or otherwiseinhibits the passage of light through the front of the display in aregion over the opening in the black matrix, then the photo TFTtransistor will be in an “off” state. If the photo TFT is “off” then thevoltage across the capacitor Cst2 will not significantly dischargethrough the photo TFT. Accordingly, the charge imposed across Cst2 willbe substantially unchanged. In essence, the voltage imposed across Cst2will remain substantially unchanged if the ambient light is inhibitedfrom striking the photo TFT.

To determine the voltage across the capacitor Cst2, a voltage is imposedon the select line which causes the gate of the readout TFT tointerconnect the imposed voltage on Cst2 to the readout line. If thevoltage imposed on the readout line as a result of activating thereadout TFT is substantially unchanged, then the output of theoperational amplifier will be substantially unchanged (e.g., zero). Inthis manner, the system is able to determine whether the light to thedevice has been inhibited, in which case the system will determine thatthe screen has been touched at the corresponding portion of the displaywith the photo TFT.

During the readout cycle, the voltage imposed on the select line causesthe voltage on the respective drain of the photo TFT and the drain ofthe readout TFT to be coupled to the respective readout line, whichresults in resetting the voltage potential across Cst2. The voltagecoupled to the drain of the photo TFT and the drain of the readout TFTis approximately ground (e.g., zero volts) with the non-inverting inputof the operational amplifier connected to ground. The voltage imposed onthe select line is removed so that the readout TFT will turn “off”. Inthis manner, the act of reading the voltage simultaneously acts to resetthe voltage potential for the next cycle.

Under normal operational conditions ambient light from the front of thedisplay passes through the black matrix and strikes the amorphoussilicon of the photo TFT. If a person does not touch the front of thedisplay in a region over the opening in the black matrix or otherwiseinhibits the passage of light through the front of the display in aregion over the opening in the black matrix, then the photo TFTtransistor will be in an “on” state. If the photo TFT is “on” then thevoltage across the capacitor Cst2 will significantly discharge throughthe photo TFT, which is coupled to the common line. In essence thevoltage imposed across Cst2 will decrease toward the common voltage.Accordingly, the charge imposed across Cst2 will be substantiallychanged in the presence of ambient light. Moreover, there is asubstantial difference in the voltage potential across the holdcapacitor when the light is not inhibited versus when the light isinhibited.

Similarly, to determine the voltage across the capacitor Cst2, a voltageis imposed on the select line which causes the gate of the readout TFTto interconnect the imposed voltage to the readout line. If the voltageimposed on the readout line as a result of activating the readout TFT issubstantially changed or otherwise results in an injection of current,then the output of the operational amplifier will be substantiallynon-zero. The output voltage of the operational amplifier isproportional or otherwise associated with the charge on the capacitorCst2. In this manner, the system is able to determine whether the lightto the device has been uninhibited, in which case the system willdetermine that the screen has not been touched at the correspondingportion of the display with the photo TFT.

Referring to FIG. 8, a layout of the active matrix layer may include thephoto TFT, the capacitor Cst2, the readout TFT in a region between thepixel electrodes. Light sensitive elements are preferably included atselected intervals within the active matrix layer. In this manner, thedevice may include touch panel sensitivity without the need foradditional touch panel layers attached to the front of the display. Inaddition, the additional photo TFT, readout TFT, and capacitor may befabricated together with the remainder of the active matrix layer,without the need for specialized processing. Moreover, the complexity ofthe fabrication process is only slightly increased so that the resultingmanufacturing yield will remain substantially unchanged. It is to beunderstood that other light sensitive elements may likewise be used. Inaddition, it is to be understood that other light sensitive electricalarchitectures may likewise be used.

Referring to FIG. 11, a graph of the photo-currents within amorphoussilicon TFTs is illustrated. Line 300 illustrates a dark ambientenvironment with the gate connected to the source of the photo TFT. Itwill be noted that the leakage currents are low and relatively stableover a range of voltages. Line 302 illustrates a dark ambientenvironment with a floating gate of the photo TFT. It will be noted thatthe leakage currents are generally low and relatively unstable over arange of voltages (significant slope). Line 304 illustrates a lowambient environment with the gate connected to the source of the photoTFT. It will be noted that the leakage currents are three orders ofmagnitude higher than the corresponding dark ambient conditions andrelatively stable over a range of voltages. Line 306 illustrates a lowambient environment with a floating gate of the photo TFT. It will benoted that the leakage currents are generally three orders of magnitudehigher and relatively unstable over a range of voltages (significantslope). Line 308 illustrates a high ambient environment with the gateconnected to the source of the photo TFT. It will be noted that theleakage currents are 4.5 orders of magnitude higher than thecorresponding dark ambient conditions and relatively stable over a rangeof voltages. Line 310 illustrates a high ambient environment with afloating gate of the photo TFT. It will be noted that the leakagecurrents are generally 4.5 orders of magnitude higher and relativelyunstable over a range of voltages (significant slope). With thesignificant difference between the dark state, the low ambient state,and the high ambient state, together with the substantially flatresponses over a voltage range (source-drain voltage), the system mayreadily process the data in a confident manner, especially with the gateconnected to the source.

Referring to FIG. 9, under high ambient lighting conditions the photoTFT will tend to completely discharge the Cst2 capacitor to the commonvoltage, perhaps with an offset voltage because of the photo TFT. Inthis manner, all of the photo TFTs across the display will tend todischarge to the same voltage level. Those regions with reduced ambientlighting conditions or otherwise where the user blocks ambient lightfrom reaching the display, the Cst2 capacitor will not fully discharge,as illustrated by the downward spike in the graph. The downward spike inthe graph provides location information related to the region of thedisplay that has been touched.

Referring to FIG. 10, under lower ambient lighting conditions the photoTFT will tend to partially discharge the Cst2 capacitor to the commonvoltage. In this manner, all of the photo TFTs across the display willtend to discharge to some intermediate voltage levels. Those regionswith further reduced ambient lighting conditions or otherwise where theuser blocks ambient light from reaching the display, the Cst2 capacitorwill discharge to a significantly less extent, as illustrated by thedownward spike in the graph. The downward spike in the graph provideslocation information related to the region of the display that has beentouched. As shown in FIGS. 9 and 10, the region or regions where theuser inhibits light from reaching the display may be determined aslocalized minimums. In other embodiments, depending on the circuittopology, the location(s) where the user inhibits light from reachingthe display may be determined as localized maximums or otherwise somemeasure from the additional components.

In the circuit topology illustrated, the value of the capacitor Cst2 maybe selected such that it is suitable for high ambient lightingconditions or low ambient lighting conditions. For low ambient lightingconditions, a smaller capacitance may be selected so that the device ismore sensitive to changes in light. For high ambient lightingconditions, a larger capacitance may be selected so that the device isless sensitive to changes in light. In addition, the dimensions of thephototransistor may be selected to change the photo-leakage current.Also, one set of light sensitive elements (e.g., the photo TFT and thecapacitance) within the display may be optimized for low ambientlighting conditions while another set of light sensitive elements (e.g.,the photo TFT and the capacitance) within the display may be optimizedfor high ambient lighting conditions. Typically, the data from lightsensitive elements for low ambient conditions and the data from lightsensitive elements for high ambient conditions are separately processed,and the suitable set of data is selected. In this manner, the samedisplay device may be used for high and low ambient lighting conditions.In addition, multiple levels of sensitivity may be provided. It is to beunderstood that a single architecture may be provided with a wide rangeof sensitivity from low to high ambient lighting conditions.

Another structure that may be included is selecting the value of thecapacitance so that under normal ambient lighting conditions the chargeon the capacitor only partially discharges. With a structure where thecapacitive charge only partially discharges, the present inventorsdetermined that an optical pointing device, such as a light wand orlaser pointer, might be used to point at the display to furtherdischarge particular regions of the display. In this manner, the regionof the display that the optical pointing device remains pointed at maybe detected as local maximums (or otherwise). In addition, those regionsof the display where light is inhibited will appear as local minimums(or otherwise). This provides the capability of detecting not only theabsence of light (e.g., touching the panel) but likewise those regionsof the display that have increased light incident thereon. Referring toFIG. 12, a graph illustrates local minimums (upward peaks) from addedlight and local maximums (downward peaks) from a lack of light. Inaddition, one set of light sensitive elements (e.g., the photo TFT andthe capacitance) within the display may be optimized for ambientlighting conditions to detect the absence of light while another set oflight sensitive elements (e.g., the photo TFT and the capacitance)within the display may be optimized for ambient lighting conditions todetect the additional light imposed thereon.

A switch associated with the display may be provided to select among aplurality of different sets of light sensitive elements. For example,one of the switches may select between low, medium, and high ambientlighting conditions. For example, another switch may select between atouch sensitive operation (absence of light) and an optical pointingdevice (addition of light). In addition, the optical pointing device maycommunicate to the display, such as through a wire or wirelessconnection, to automatically change to the optical sensing mode.

It is noted that the teachings herein are likewise applicable totransmissive active matrix liquid crystal devices, reflective activematrix liquid crystal devices, transflective active matrix liquidcrystal devices, etc. In addition, the light sensitive elements maylikewise be provided within a passive liquid crystal display. Thesensing devices may be, for example, photo resistors and photo diodes.

Alternatively, light sensitive elements may be provided between the rearpolarizing element and the active matrix layer. In this arrangement, thelight sensitive elements are preferably fabricated on the polarizer, orotherwise a film attached to the polarizer. In addition, the lightsensitive elements may be provided on a thin glass plate between thepolarizer and the liquid crystal material. In addition, the black matrixor otherwise light inhibiting material is preferably arranged so as toinhibit ambient light from striking the readout TFT while free frominhibiting light from striking the photo TFT. Moreover, preferably alight blocking material is provided between the photo TFT and/or thereadout TFT and the backlight, such as gate metal, if provided, toinhibit the light from the backlight from reaching the photo TFT and/orthe readout TFT.

Alternatively, light sensitive elements may be provided between thefront polarizing element and the liquid crystal material. In thisarrangement, the light sensitive elements are preferably fabricated onthe polarizer, or otherwise a film attached to the polarizer. Inaddition, the light sensitive elements may be provided on a thin glassplate between the polarizer and the liquid crystal material. The lightsensitive elements may likewise be fabricated within the front electrodelayer by patterning the front electrode layer and including suitablefabrication techniques. In addition, a black matrix or otherwise lightinhibiting material is preferably arranged so as to inhibit ambientlight from striking the readout TFT while free from inhibiting lightfrom striking the photo TFT. Moreover, preferably a light blockingmaterial is provided between the photo TFT and/or the readout TFT andthe backlight, if provided, to inhibit the light from the backlight fromreaching the photo TFT and/or the readout TFT.

Alternatively, light sensitive elements may be provided between thefront of the display and the rear of the display, normally fabricated onone of the layers therein or fabricated on a separate layer providedwithin the stack of layers within the display. In the case of a liquidcrystal device with a backlight the light sensitive elements arepreferably provided between the front of the display and the backlightmaterial. The position of the light sensitive elements are preferablybetween (or at least partially) the pixel electrodes, when viewed from aplan view of the display. This may be particularly useful for reflectivedisplays where the pixel electrodes are opaque. In addition forreflective displays, any reflective conductive electrodes should bearranged so that they do not significantly inhibit light from reachingthe light sensitive elements. In this arrangement, the light sensitiveelements are preferably fabricated on one or more of the layers, orotherwise a plate attached to one or more of the layers. In addition, ablack matrix or otherwise light inhibiting material is preferablyarranged so as to inhibit ambient light from striking the readout TFTwhile free from inhibiting light from striking the photo TFT. Moreover,preferably a light blocking material is provided between the photo TFTand/or the readout TFT and the backlight, if provided, to inhibit thelight from the backlight from reaching the photo TFT and/or the readoutTFT.

In many applications it is desirable to modify the intensity of thebacklight for different lighting conditions. For example, in darkambient lighting conditions it may be beneficial to have a dimbacklight. In contrast, in bright ambient lighting conditions it may bebeneficial to have a bright backlight. The integrated light sensitiveelements within the display stack may be used as a measure of theambient lighting conditions to control the intensity of the backlightwithout the need for an additional external photo-sensor. One lightsensitive element may be used, or a plurality of light sensitive elementmay be used together with additional processing, such as averaging.

In one embodiment, the readout line may be included in a periodic mannerwithin the display sufficient to generally identify the location of the“touch”. For example the readout line may be periodically added at each30^(th) column. Spacing the readout lines at a significant number ofpixels apart result in a display that nearly maintains its previousbrightness because most of the pixel electrodes have an unchanged size.However, after considerable testing it was determined that such periodicspacing results in a noticeable non-uniform gray scale because ofdifferences in the size of the active region of the pixel electrodes.One potential resolution of the non-uniform gray scale is to modify theframe data in a manner consistent with the non-uniformity, such asincreasing the gray level of the pixel electrodes with a reduced size orotherwise reducing the gray levels of the non-reduced size pixelelectrodes, or a combination thereof. While a potential resolution, thisrequires additional data processing which increases the computationalcomplexity of the system.

A more desirable resolution of the non-uniform gray scale is to modifythe display to include a readout line at every third pixel, or otherwisein a manner consistent with the pixel electrode pattern of the display(red pixel, green pixel, blue pixel). Alternatively, a readout line isincluded at least every 12^(th) pixel (36 pixel electrodes of a red,blue, green arrangement), more preferably at least every 9^(th) pixel(27 pixel electrodes of a red, blue, green arrangement), even morepreferably at least every 6^(th) pixel (18 pixel electrodes of a red,blue, green arrangement or 24 pixel electrodes of a red, blue, bluegreen arrangement), and most preferably at least every 3^(rd) pixel (3pixel electrodes of a red, blue, green arrangement). The readout linesare preferably included for at least a pattern of four times the spacingbetween readout lines (e.g., 12^(th) pixel times 4 equals 48 pixels,9^(th) pixel times 4 equals 36 pixels). More preferably the pattern ofreadout lines is included over a majority of the display. The resultingdisplay may include more readout lines than are necessary to accuratelydetermine the location of the “touch”. To reduce the computationalcomplexity of the display, a selection of the readout lines may be freefrom interconnection or otherwise not operationally interconnected withreadout electronics. In addition, to further reduce the computationalcomplexity of the display and to increase the size of the pixelelectrodes, the readout lines not operationally interconnected withreadout electronics may likewise be free from an associated lightsensitive element. In other words, additional non-operational readoutlines may be included within the display to provide a gray scale displaywith increased uniformity. In an alternative embodiment, one or more ofthe non-operational readout lines may be replaced with spaces. In thismanner, the gray scale display may include increased uniformity, albeitwith additional spaces within the pixel electrode matrix.

The present inventors considered the selection of potential pixelelectrodes and came to the realization that the electrode correspondingto “blue” light does not contribute to the overall white transmission tothe extent that the “green” or “red” electrodes. Accordingly, the systemmay be designed in such a manner that the light sensitive elements areassociated with the “blue” electrodes to an extent greater than theirassociation with the “green” or “red” electrodes. In this manner, the“blue” pixel electrodes may be decreased in size to accommodate thelight sensitive elements while the white transmission remainssubstantially unchanged. Experiments have shown that reducing the sizeof the “blue” electrodes to approximately 85% of their original size,with the “green” and “red” electrodes remaining unchanged, results in areduction in the white transmission by only about 3 percent.

While such an additional set of non-operational readout lines providesfor increased uniform gray levels, the reduction of pixel aperturesresults in a reduction of brightness normally by at least 5 percent andpossibly as much as 15 percent depending on the resolution and layoutdesign rules employed. In addition, the manufacturing yield is decreasedbecause the readout line has a tendency to short to its neighboring dataline if the processing characteristics are not accurately controlled.For example, the data line and readout line may be approximately 6-10microns apart along a majority of their length.

Referring to FIG. 13, to increase the potential manufacturing yield andthe brightness of the display, the present inventors came to therealization that the readout of the photo-sensitive circuit and thewriting of data to the pixels may be combined on the same bus line, orotherwise a set of lines that are electrically interconnected to oneanother. To facilitate the use of the same bus line, a switch 418 mayselect between providing new data 420 to the selected pixels and readingdata 414 from the selected pixels. With the switch 418 set tointerconnect the new data 420 with the selected pixels, the data from aframe buffer or otherwise the video data stream may be provided to thepixels associated with one of the select lines. Multiple readoutcircuits may be used, or one or more multiplexed readout circuits maybeused. For example, the new data 420 provided on data line 400 may be 4.5volts which is latched to the pixel electrode 402 and the photo TFT 404by imposing a suitable voltage on the select line 406. In this manner,the data voltage is latched to both the pixel electrode and acorresponding photo-sensitive circuit.

The display is illuminated in a traditional manner and the voltageimposed on the photo TFT 404 may be modified in accordance with thelight incident on the photo-sensitive circuit, as previously described.In the topology illustrated, the photo TFT 404 is normally a N-typetransistor which is reverse biased by setting the voltage on the commonline 408 to a voltage lower than an anticipated voltage on the photo TFT404, such as −10 or −15 volts. The data for the current frame may bestored in a frame buffer for later usage. Prior to writing the data foranother frame, such as the next frame, the data (e.g., voltage) on thereadout TFT 410 is read out. The switch 418 changes between the new data420 to the readout line 414 interconnected to the charge readoutamplifier 412. The select line 406 is again selected to couple theremaining voltage on the photo TFT 404 through the readout TFT 410 tothe data line 400. The coupled voltage (or current) to the data line 400is provided as an input to the charge readout amplifier 412 which iscompared against the corresponding data from the previous frame 422,namely, the voltage originally imposed on the photo TFT 404. Thedifference between the readout line 414 and the data from the previousframe 422 provides an output to the amplifier 412. The output of theamplifier 412 is provided to the processor. The greater the drain of thephoto TFT 404, normally as a result of sensing light, results in agreater output of the amplifier 412. Referring to FIG. 14, an exemplarytiming for the writing and readout on the shared data line 400 isillustrated.

At low ambient lighting conditions and at dark lighting conditions, theintegrated optical touch panel is not expected to operate well to thetouch of the finger because there will be an insufficient (or none)difference between the signals from the surrounding area and the touchedarea. To alleviate the inability to effectively sense at the low anddark ambient lighting conditions a light pen or laser pointer may beused (e.g., light source), as previously described. The light source maybe operably interconnected to the display such as by a wire or wirelesscommunication link. With the light source operably interconnected to thedisplay the intensity of the light source may be controlled, at least inpart, by feedback from the photo-sensitive elements or otherwise thedisplay, as illustrated in FIG. 15. When the display determines thatsufficient ambient light exists, such as ambient light exceeding athreshold value, the light source is turned “off”. In this manner,touching the light source against the display results in the same effectas touching a finger against the display, namely, impeding ambient lightfrom striking the display. When the display determines that insufficientambient light exists, such as ambient light failing to exceed athreshold value, the light source is turned “on”. In this manner,touching or otherwise directing the light from the light source againstthe display results in a localized increase in the received lightrelative to the ambient light level. This permits the display to beoperated in dark ambient lighting conditions or by feedback from thedisplay. In addition, the intensity of the light from the light sourcemay be varied, such as step-wise, linearly, non-linearly, orcontinuously, depending upon the ambient lighting conditions.Alternatively, the light source may include its own ambient lightdetector so that feedback from the display is unnecessary and likewisecommunication between the light source and the display may beunnecessary.

While using light from an external light source while beneficial it maystill be difficult to accurately detect the location of the additionallight because of background noise within the system and variablelighting conditions. The present inventors considered this situation anddetermined that by providing light during different frames, such as oddframes or even frames, or odd fields or even fields, or every thirdframe, or during selected frames, a more defined differential signalbetween the frames indicates the “touch” location. In essence, the lightmay be turned on and off in some manner, such as blinking at a ratesynchronized with the display line scanning or frames. An exemplarytiming for an odd/even frame arrangement is shown in FIG. 16. Inaddition, the illumination of some types of displays involves scanningthe display in a row-by-row manner. In such a case, the differentialsignal may be improved by modifying the timing of the light pulses inaccordance with the timing of the gate pulse (e.g., scanning) for therespective pixel electrodes. For example, in a top-down scanning displaythe light pulse should be earlier when the light source is directedtoward the top of the display as opposed to the bottom of the display.The synchronization may be based upon feedback from the display, ifdesired.

In one embodiment, the light source may blink at a rate synchronizedwith the display line scanning. For example, the light source may usethe same driver source as the image pixel electrodes. In anotherembodiment the use of sequential (or otherwise) frames may be subtractedfrom one another which results in significant different between signaland ambient conditions. Preferably, the light sensitive elements have adynamic range greater than 2 decades, and more preferably a dynamicrange greater than 4 decades. If desired, the system may use twosequential fields of scanning (all lines) subtracted from the next twofields of scanning (all lines) so that all the lines of the display areused.

Another technique for effective operation of the display in dark or lowlevel ambient conditions is using a pen or other device with a lightreflecting surface that is proximate (touching or near touching) thedisplay when interacting with the display. The light from the backlighttransmitted through the panel is then reflected back into thephoto-sensitive element and the readout signal will be greater at thetouch location than the surrounding area.

Referring to FIG. 17, another type of reflective liquid crystal display,typically used on handheld computing devices, involves incorporating alight guide in front of the liquid crystal material, which is normally aglass plate or clear plastic material. Normally, the light guide isconstructed from an opaque material having an index of refractionbetween 1.4 and 1.6, more typically between 1.45 and 1.50, and sometimesof materials having an index of refraction of 1.46. The light guide isfrequently illuminated with a light source, frequently disposed to theside of the light guide. The light source may be any suitable device,such as for example, a cold cathode fluorescent lamp, an incandescentlamp, and a light emitting diode. To improve the light collection areflector may be included behind the lamp to reflect light that isemitted away from the light guide, and to re-direct the light into thelight guide. The light propagating within the light guide bouncesbetween the two surfaces by total internal reflections. The totalinternal reflections will occur for angles that are above the criticalangle, measured relative to the normal to the surfaces, as illustratedin FIG. 18. To a first order approximation, the critical angle β maybedefined by Sin(β)=1/n where n is the index of refraction of the lightguide. Since the surfaces of the light guide are not perfectly smooththere will be some dispersion of the light, which causes someillumination of the display, as shown in FIG. 19.

The present inventors came to the realization that the critical angleand the disruption of the total internal reflections may be modified insuch a manner as to provide a localized increase in the diffusion oflight. Referring to FIG. 20, one suitable technique for the localizeddiffusion of light involves using a plastic pen to touch the front ofthe display. The internally reflected light coincident with the locationthat the pen touches the display will significantly diffuse and bedirected toward the photo sensitive elements within the display. Theplastic pen, or other object including the finger or the eraser of apencil, preferably has an index of refraction within 0.5, morepreferably within 0.25, of the index of refraction of the light guide.For example, the index of refraction of the light guide may be between1.2 and 1.9, and more preferably between 1.4 and 1.6. With the twoindexes of refraction sufficiently close to one another the disruptionof the internal reflections, and hence amount of light directed towardthe photo-sensitive elements, is increased. In addition, the plastic penpreferably has sufficient reflectivity of light as opposed to beingnon-reflective material, such as for example, black felt.

Referring to FIG. 21, after further consideration the present inventorswere surprised to note that a white eraser a few millimeters away fromthe light guide results in a darkened region with generally consistentoptical properties while a white eraser in contact with the light guideresults in a darkened region with generally consistent opticalproperties together with a smaller illuminated region. In the preferredembodiment, the light sensitive elements are positioned toward the frontof the display in relation to the liquid crystal material (or otherwisethe light valve or electroluminescent material) so that a clearer imagemay be obtained. It is to be understood that any suitable pointingdevice may be used. The illuminated region has an illumination brighterin relation to the remainder of the darkened region. The illuminatedregion may be located by any suitable technique, such as for example, acenter of gravity technique.

After further consideration of the illuminated region the presentinventors came to the realization that when users use a “touch panel”display, there is a likelihood that the pointing device (or finger) may“hover” at a location above the display. Normally, during this hoveringthe user is not actually selecting any portion of the display, butrather still deciding where to select. In this manner, the illuminatedregion is beneficial because it provides a technique for thedetermination between when the user is simply “hovering” and the userhas actually touched (e.g., “touching”) the display.

Another potential technique for the determination between “hovering” and“touching” is to temporally model the “shadow” region (e.g., lightimpeded region of the display). In one embodiment, when the user istypically touching the display then the end of the shadow will typicallyremain stationary for a period of time, which may be used as a basis, atleast in part, of “touching”. In another embodiment, the shadow willtypically enlarge as the pointing device approaches the display andshrinks as the pointing device recedes from the display, where thegeneral time between enlarging and receding may be used as a basis, atleast in part, of “touching”. In another embodiment, the shadow willtypically enlarge as the pointing device approaches the display andmaintain the same general size when the pointing device is touching thedisplay, where the general time where the shadow maintains the same sizemay be used as a basis, at least in part, of “touching”. In anotherembodiment, the shadow will typically darken as the pointing deviceapproaches the display and maintain the same shade when the pointingdevice is touching the display, where the general time where the shadowmaintains the same general shade may be used as a basis, at least inpart, of “touching”.

While attempting to consider implementation of such techniques on ahandheld device it came to the inventor's surprise that the displayportion of a handheld device has a refresh rate generally less than therefresh rate of the portion of the handwriting recognition portion ofthe display. The handheld portion of the display may use any recognitiontechnique, such as Palm OS™ based devices. The refresh rate of thedisplay is typically generally 60 hertz while the refresh rate of thehandwriting portion of the display is typically generally 100 hertz.Accordingly, the light-sensitive elements should be sampled at asampling rate corresponding with the refresh rate of the respectiveportion of the display.

The technique described with respect to FIG. 20 operates reasonably wellin dark ambient lighting conditions, low ambient lighting conditions,regular ambient lighting conditions, and high ambient lightingconditions. During regular and high ambient lighting conditions, thedisplay is alleviated of a dependency on the ambient lightingconditions. In addition, with such lighting the illumination point ismore pronounced and thus easier to extract. Unfortunately, during thedaytime the ambient light may be sufficiently high causing the detectionof the pointing device difficult. In addition, shades of the ambientlight may also interfere with the detection techniques.

The present inventors considered improving the robustness of thedetection techniques but came to the realization that with sufficient“noise” in the system the creation of such sufficiently robusttechniques would be difficult. As opposed to the traditional approach ofimproving the detection techniques, the present inventors came to therealization that by providing light to the light guide of a limitedselection of wavelengths and selectively filtering the wavelengths oflight within the display the difference between touched and un-touchedmay be increased. As an initial matter the light from the light sourceprovided to the light guide is modified, or otherwise filtered, toprovide a single color. Alternatively, the light source may providelight of a range of wavelengths, such as 600-700 nm, or 400-500 and530-580, or 630. Typically, the light provided to the light guide has arange of wavelengths (in any significant amount) less than white lightor otherwise the range of wavelengths of ambient light. Accordingly,with the light provided to the light guide having a limited color gamut(or reduced color spectrum) the touching of the pointing device on thedisplay results in the limited color gamut light being locally directedtoward the light-sensitive elements. With a limited color gamut lightbeing directed toward the display as a result of touching the lightguide (or otherwise touching the front of the display), a color filtermay be included between the light guide and the light-sensitive elementsto filter out at least a portion of the light not included within thelimited color gamut. In other words, the color filter reduces thetransmission of ambient light to an extent greater than the transmissionof light from the light source or otherwise within the light guide. Forexample, the ambient light may be considered as “white” light while thelight guide has primarily “red” light therein. A typical transmission ofa red color filter for ambient white light may be around 20%, while thesame color filter will transmit about 85% of the red light. Preferablythe transmission of ambient light through the color filter is less than75% (greater than 25% attenuation) (or 60%, 50%, 40%, 30%) while thetransmission of the respective light within the light guide is greaterthan 25% (less than 25% attenuation) (or 40%, 50%, 60%, 70%), so that inthis manner there is sufficient attenuation of selected wavelengths ofthe ambient light with respect to the wavelengths of light within thelight guide to increase the ability to accurately detect the touching.

In another embodiment, the light source to the light guide may include aswitch or otherwise automatic modification to “white” light whenoperated in low ambient lighting conditions. In this manner, the displaymay be more effective viewed at low ambient lighting conditions.

In another embodiment, the present inventors determined that if thelight source providing light to the display was modulated in somefashion an improvement in signal detection may be achieved. For example,a pointing device with a light source associated therewith may modulatethe light source in accordance with the frame rate of the display. Witha frame rate of 60 hertz the pointing device may for example modulatethe light source at a rate of 30 hertz, 20 hertz, 10 hertz, etc. whichresults in additional light periodically being sensed by the lightsensitive elements. Preferably, the light source is modulated(“blinked”) at a rate synchronized with the display line scanning, anduses the same raw drivers as the image thin-film transistors. Theresulting data may be processed in a variety of different ways.

In one embodiment, the signals from the light sensitive elements areused, as captured. The resulting improvement in signal to backgroundratio is related to the pulse length of the light relative to the frametime. This provides some additional improvement in signal detectionbetween the light generated by the pointing device relative to theambient light (which is constant in time).

In another embodiment, multiple frames are compared against one anotherto detect the presence and absence of the additional light resultingfrom the modulation. In the case of subsequent frames (sequential ornon-sequential), one without additional light and one with additionallight, the data from the light sensitive elements may be subtracted fromone another. The improvement in signal to background ratio is related tothe periodic absence of the additional light. In addition, thisprocessing technique is especially suitable for low ambient lighting andhigh ambient lighting conditions. Preferably the dynamic range of thesensors is at least 4 decades, and two sequential frames with additionallight and two sequential frames without additional light are used sothat all of the scanning lines are encompassed. When the system chargesa sensor it takes a whole frame for it to discharge by the light. Sincethe first line will start at time zero and take a frame time, the lastline will be charged after almost a frame and will take another frametime to discharge. Therefore, the system should preferably use twoframes with additional illumination and then two frames withoutadditional illumination.

All references cited herein are hereby incorporated by reference.

The terms and expressions which have been employed in the foregoingspecification are used therein as terms of description and not oflimitation, and there is no intention, in the use of such terms andexpressions, of excluding equivalents of the features shown anddescribed or portions thereof, it being recognized that the scope of theinvention is defined and limited only by the claims which follow.

1. A light sensitive display comprising: (a) a light valve including afront polarizing element, a rear polarizing element, and light rotatingmaterial located between said first polarizing element and said rearpolarizing element; (b) a plurality of light sensitive elements locatedbetween said front polarizing element and said light rotating material;(c) readout system distinguished between ambient light and another lightlevel sensed by at least one of said light sensitive elements.